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ABSTRACT 

In a post mortem study^ it is demonstrated that 
linear prediction is as effective as computing a negative 
hyper-gecmetric distribution for estirnating test norms following 
matrax sampling from a total test with a highly skewed score 
distribution^ provided the same prediction coefficient is used for 
all examinee groups. It is also demonstrated empirically and 
algebraically that using a coefficient unique to each examinee group 
produces distributions of predicted total-test scores with 
••insufficient" variance. Implications for measurement practice and 
statistical theory are discussed^ (Authors) 
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Abstract 



Id a post. mortem study, it is demonitrated that linear 
prediqtipn is as effeetive ae computing a negative hyper* 
geoaetrlG distribution for eatiinatlng teet norffis followiflg 
mtTix fiafflpling from a total teet with a highly aktwed icore 
distribution, provided the samt prediGtion eoeffielent ii 
used for all examinee groups* It is also revealed that using 
a cotf fieient unique to each e»mlnte group produces difltribu- 
tions of pridlcted total-test scorei with "insuf fieient" 
variance , 



Klelnke (1972) hai preiented a technique tor generating total- 
teit score distributioni fDlloving matrix sampling. After txaHlntt 
reiponds to a sample of the items, hii score on the Items with whiah 
he MB not presented is predicted. In the present paper, results of 
two further inveiti^tions of the tichnique are prgeented. 

Matrix sai^ling is that prooedurt under which examineei and IttM 
are ilfflultaneouily lan^led* Each axaminee^ then, responds to only a 
sample of items from the total test. The advantage of this procedure 
for test norming, as first presentid by Lord (1962), is that by short- 
ening the amoimt of time each exarainte rauit i^nd In the norming proeess^ 
cooperation of the examineei and their supervlsore Is enhanced and the 
repreientativeiieai of the normi thus obtained is enlULneed* A number of 
studiei of mtrix iampling have appeared^ demonitrating that accurate 
estissates of the total-tesL mtan and variance are possible. Of these 
*udiei,in only a few (e.g., Cooke 4 Stufflebiam, 19671 Klelnke^ 19721 
Lord, 1962) has the total-test score distribution been estiBBpted, While 
the mean and the variance are obviously essential data. It is equally 
obvious that inforrsation about the overall diitribution is at leaat 
helpful in test norming* 

The technique suggested by Lord (1962) for generating a total-teit 
diitribution is to use the estimattd mtan and TOriance and the number of 
items in the totel test tc generate a ne^tive hy^rgeometric distribution 
(H) for the scores* Klelnke 'i (1972) approaah is bmsed on total-ttit score 
eitlmates for the examines* It uses a licear-prediGtion (lp) equation 



(2) 

to prediet a score for each examinee on the compoiitt of itemi ¥ith which 
he w&i not prestntedi 

vhef ^ 

y| is examinee i*s score on all items not in subtest X, 
whieh is the iubteit to whleh i responded, 

Sj ii the estimated total-^test standard deviation, 

is the standard deviation of subteit X, 

is i's score ©n Xj 

X is the mean of and 

y li the eitirMted rotan of composite Y. 

An txafflinee^g predleted total-teet seore, is merely the sum of X^ and , 

In Kleinke (1972), it was defflonitrated that the distribution following 

U was as adequate as that of a generated H| for the total test he used. That 

teit had an underlying score distribution that was only slightly ^eitively 

skewed, but was subsequently dlicovired to have an underlying true-seora 

distribution that did not go to Eero, beeauie of the operation of chance 

success* For thlE reaion, the H-generated dlitribution was systenflatically, 

but slightly, different from the criterion distribution,^ At the same time, 

the distribution had sTOll and non-systematic differences from the cr iter Ion 

distribution cauaad by the rounding of predicted scores to integers and the 

resulting phenomenon that, for many scores, no examinees were predicted. 

Two questions raised in Kleinke (1972) were inveiti^ted in the resent 

study, ^ First, could be used if the total-test score dietribution were 

highly skewed? Second, what would be the effect of using ij^^^, the estl- 

inatad total-test standard deviation for examinee -sample x, in placi of ' 
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Sji the mean of these eitlmatis, in Equation 1? 

Skewed Distribution 

Method 

Item- and eraminee-samples were drmm from the popuLationg used by 
Shoemaker (1970), As can be ssen in Figure 1, the test vai a twenty- 
item teat vith a J-shaped score distribution. Four Item iamplea of five 
items each were seleeted randomly without replacement. The examinees 
were randomly aailgned to one of four nonoverlapping eMminee- groups, 
each containing 257 examinees. The procedurei of Lord (1962) were used 
to obtain eatimatea of total-tait mian and imr lance. Following this, 
toth H and LP were used to approxiimte the total-test score distribution, 
Regults ^ and Dlscuislon 

The actual maan and wr lance were 17.5^ and 9*01, respeotively. The 
correaponding matrix- sampling- based eitlnates were 17*6o and 10*07, 
Cumu^tlve distribution curves are presented in Figure 1, Difftrences in 
proportions are preienttd in Table 1, together with the data froin Kltinke'i 
(1972) study, for comparative purpostiu 

AffAn/ as In the original study, LP estimates suffer from the arbitrary 
rounding* Because of this rounding, no examinees were predicted to reeelve raw 
scores of 10, 13, ifi, or 16, If even the flrit declMl wer€ retained, the 
eumulative diitribution would be much imoother and closer to the actual 
dlatributlon. However, the original decision to concentrate on Integer 
scores was baied on two conslderatiohi. First, since the application H 
provides for rounding estimated frequencies to the nearest examinee, it 
appeared reaionable to round IJ^gmierated estimates to the nearest integer. 
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Table 1. 

Maximum and raean differences betveen generated and eriterlon 
cumulative frequency distributions * 



Diitrlbutlon 



Skeved SyametrlQ 
Jiffore.nae LP H LP H 

Maximum ,131 .076 .035 .OUl 

Mean (Alg,) ^.001 " .002 ,003 .00k 

MtM (Aba,) ,021 .010 ,010 .013' 

Oivan the present retio of examinees to number of Items in the total-teat^ 
this eonaideratioa ieems to be inappropriately reitrictive*. However, the 
seoond eoniideration is still sufficiently strong to argue for eontlnu&tion 
of the prsotice of rounding UP-scores. That is becauee the advantage of 
using LP over geDeratlng H lies in Its potential for giving information to 
an exaoisee or to one who must nake a decision on the bag is of performanca 
on only part of a set of icorea, as in the case when a job applicant, for 
inatonce, has been presented vith only a portion of a regular battery of 
teati^ but the total score is used for deelsion-makingp Expressing 
txp^etid dtQinial acores on tests composed of blMrily-seored items could 
lead to confusion and the appearance of greater precision than the dato 
probably warrant. 

In addition, it should be pointed out that the greatest discrepancies 
Mre ©bsirved close to the modal wlut. This w.s also true for tht (niarly) 
iymmctria diatrlbution* On first consideration, this vould seem to be-a 
mere serious problem than it probably Is, In practice, the test constructor 
ii advised to fit the difficulty level of a test and, hence, the slmpe of 
its score distribution^ to the purpose for which tht test is intendid* 
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A highly skewed distribution^ such ai that of thii test, would b€ 
appropriate where differtneei among low georeri were relatively more 
important than identifying differencts among hlah eeorers* 

InitUlly, it vas euggested tl»t using a„(„^ in place of s in 
Equation 1 would be more reaionable for predicting total^test ioores 
from item-sample InforiDation* The rationale for that la that t j ^ 
ii baaed on the unique information that ariies from the respgniei of 
mafflbers of examinee -p-oup X and the partioular set of itemi with whioh 
they ir^re presented. 

However^ when S^^^^j was ueed^ the variancei of the predicted total- 
teat scores were all less than t^. Seme reflection and numerical and 
algebraic manipulation revealed wi^. Using %f^^^ ii eiientiany equi^ 
valent t© ayeraging (and hencej summing) essentially non-additive standard 
deylations. 



Footnote e 

The author is indebted to Frederic M, Lord vho pointed 
out the true-iaore distribution and its effect on 
After exafTiining the data. 

The author viehes to expreis his thanks to David M. 
Shoenaker for providing hm with data. 
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